Skip to content

fix: replace implicit s3fs usage with explicit S3Downloader in feature_utils - #6596

Open
pravali96 wants to merge 4 commits into
aws:masterfrom
pravali96:fix/replace-s3fs-with-s3downloader
Open

pravali96 wants to merge 4 commits into
aws:masterfrom
pravali96:fix/replace-s3fs-with-s3downloader

Conversation

@pravali96

@pravali96 pravali96 commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Issue #, if available:
s3fs needs aiobotocore, which pins botocore<1.43.118, while this package requires botocore>=1.43.127.
pip resolves that by backtracking to s3fs 0.4.2 (2020) — fsspec warns on load that it's "very old and known to cause severe performance issues." The dependency was forcing a six-year-old package into every install.

Description of changes:
Removes s3fs from sagemaker-mlops/pyproject.toml and keeps s3:// input working exactly as before — no new feature, no promotion:

  • s3:// still works. Now via S3Downloader/boto3 (already a hard dependency) instead of s3fs.
  • Gzipped objects still work. s3://b/data.csv.gz kept parsing because compression is inferred from the key. Without this it raised UnicodeDecodeError — the one real CX break, now fixed and tested.
  • Caller's credentials are used, not the default chain.
  • Docstring just notes the path may be an s3:// URI. Function stays unexported and out of the docs site, as before.

Net diff across three commits: pyproject.toml −1, feature_utils.py +41/−13, test_feature_utils.py +158.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…e_utils

- Add explicit S3 path detection using dataframe_or_path.startswith('s3://')
- Use S3Downloader.read_bytes() with io.BytesIO() wrapper for S3 paths
- Remove s3fs dependency from sagemaker-mlops pyproject.toml
- Preserves existing functionality for DataFrames and local file paths

Testing:
- All 43 existing unit tests pass - no regressions
- Added test_s3fs_equivalence.py proving S3Downloader.read_bytes produces
  IDENTICAL results to s3fs for the same S3 input
- Equivalence test validates: same DataFrame shape, columns, dtypes, and values
- See EQUIVALENCE_PROOF.md for detailed analysis
Follow-up to the s3fs removal in prepare_fg_from_dataframe_or_file.

Three issues with the previous commit:

1. The S3 read dropped the caller's credentials. S3Downloader.read_bytes was
   called without a session, so read_bytes fell back to `sagemaker_session or
   Session()` and built a fresh default-credential-chain session, while the
   FeatureGroup was built from the caller's session resolved ~40 lines later.
   A caller passing an explicit cross-account session read S3 as the ambient
   identity. Session resolution now happens before any data is read and is
   passed through to read_bytes.

2. pathlib.Path inputs were rejected. The signature accepts
   Union[str, Path, DataFrame] but the branch tested only `str`, so a Path fell
   through to the "Invalid type" error. The branch now accepts both and the
   error message lists the real accepted types.

3. EQUIVALENCE_PROOF.md and test_s3fs_equivalence.py were committed to the
   repository root. The script was a standalone main() that manipulated
   sys.path and asserted pd.read_csv(BytesIO(x)) == pd.read_csv(BytesIO(x)) --
   the same expression on both sides, so it exercised neither s3fs nor the new
   branch. Both files are removed and replaced with real unit tests.

Tests added to the existing
sagemaker-mlops/tests/unit/sagemaker/mlops/feature_store/test_feature_utils.py:

- an s3:// path is fetched via S3Downloader and the downloaded bytes (not the
  URI) are what reach load_feature_definitions
- the caller's session is threaded into read_bytes, for both an explicit
  session and one resolved from role + region
- a local path never touches S3Downloader
- pathlib.Path is accepted

Three of the four fail against the previous commit's source and pass against
this one. Full file: 47 passed, 24 failed; the 24 failures are identical at
5fe6647 (before any s3fs work), so they are pre-existing and unrelated.

Note for reviewers: dropping s3fs is a behavioural change for callers that
passed an s3:// path together with fsspec `storage_options` in **kwargs --
those kwargs were previously forwarded to s3fs and will now raise against a
BytesIO buffer.
---
X-AI-Prompt: Remove s3fs from the SageMaker SDK; investigate why it was introduced and what used it, then fix the session-threading and Path gaps in PR aws#6596 and replace the stray proof files with real unit tests.
X-AI-Tool: Claude Code
Preserves existing behaviour for s3:// input after the s3fs removal.

pandas infers compression from the file extension only when read_csv is given
a path. The s3:// branch hands it an in-memory buffer instead, which has no
filename, so inference was lost and a gzipped object failed:

    pd.read_csv(io.BytesIO(gzip.compress(csv)))
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1

s3://bucket/data.csv.gz worked before this change (pandas saw the URL and
inferred gzip from the suffix), so this would have been a silent regression.
Compression is now inferred from the S3 key with pandas' own
infer_compression, covering .gz/.bz2/.zip/.xz/.zst. An explicit compression=
from the caller still wins.

The docstring now mentions that the path may be an s3:// URI -- stating what
already worked, not promoting S3 input to a documented feature; the function
remains unexported and absent from the docs site, as before.

Tests (added to the existing test_feature_utils.py):
- a gzipped S3 object parses; fails without the inference, verified
- an explicit compression= kwarg is not overridden by inference

Class: 21 passed. File: 49 passed, 24 failed -- those 24 are a local
config.yaml missing SchemaVersion (jsonschema validation in
sagemaker/core/config/config.py), identical on master and unrelated to s3fs.

Known difference, not addressed here: read_bytes buffers the whole object
before parsing where s3fs streamed, so peak memory is higher on large CSVs
and chunksize=/iterator= no longer stream the download.
---
X-AI-Prompt: Remove s3fs from sagemaker-mlops without changing the customer experience; keep s3:// input working as before and don't promote S3 to a first-class feature.
X-AI-Tool: Claude Code
storage_options configures the fsspec filesystem pandas would have opened an
s3:// URI with -- for S3 that was s3fs.S3FileSystem(**storage_options), giving
callers anon, requester_pays, client_kwargs/endpoint_url and friends.

The object is now fetched with S3Downloader before parsing, so read_csv
receives a buffer and rejects the argument outright:

    ValueError: storage_options passed with file object or non-fsspec file path

Nothing in this package passes storage_options; it can only arrive from a
caller, since **kwargs are forwarded to read_csv and the docstring points at
the pandas read_csv reference. Surfacing a pandas error about file objects is
not useful, so the argument is dropped with a warning via core's existing
pop_out_unused_kwarg helper ("storage_options supplied in kwargs will be
ignored") and the read proceeds on the session's credentials.

Only the s3:// branch drops it -- for a local path storage_options remains
pandas' business and is still forwarded.

Note this does remove capability rather than relocate it: anon,
requester_pays and a custom endpoint_url have no S3Downloader equivalent.
Callers needing specific credentials should pass a configured session=.

Tests:
- storage_options on an s3:// path warns and the read still succeeds; fails
  with the pandas ValueError if the pop is removed, verified
- storage_options on a local path still reaches read_csv untouched

Class: 23 passed. File: 51 passed, 24 failed -- the 24 are a local
config.yaml missing SchemaVersion, identical on master and unrelated.

Re-verified end to end against a dev account (us-west-2): s3:// plain and
gzipped reads on both a default and a non-default bucket, plus local path,
Path and DataFrame inputs. 8/8.
---
X-AI-Prompt: Follow the SDK's own convention for a no-longer-applicable kwarg -- warn and drop storage_options on the s3:// path instead of letting pandas raise, with tests.
X-AI-Tool: Claude Code
@pravali96
pravali96 marked this pull request as ready for review November 19, 2026 23:23

This branch is waiting to be deployed

1 waiting deployment
manual-approval — 8c7381ec Waiting Nov 19, 2026 by pravali96 via wait-for-approval #908
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant